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Abstract —The road-tire friction coefficient /x is as fun¬ 
damental information for the algorithms dealing with the 
vehicle dynamics with high accuracy like in emergency cases. 
This paper introduces a new predictive methodology for the 
estimation of /x by using a camera and a microphone. After a 
description of the limits of the current methodologies, the new 
concept will be described step-by-step by following the data 
flow. The algorithm extracts the patterns corresponding of the 
different /x depending on the general luminance. These patterns 
will be matched on the current specimens to deduce the friction 
coefficient along the road ahead and a confidence value. Finally 
the results will be auto-correlated over the time to improve 
their stability. Moreover the reliability will be improved over 
a correlation with local measures based on microphone. 

The estimation of the road-tire friction coefficient // is 
fundamental for an adequate computation of the vehicle 
dynamics. Therefore different approaches have been 
investigated during the European SPARC project; and 
the method presented here has been chosen finally to be 
integrated on both truck and passenger car prototypes. 

I. Needs of a new RTFE metholodgy 

Several methods to estimate /x are based on the vehicle 
state where internal sensors are used by an auxiliary function 
conjointly to a dynamic vehicle model. The actual state 
of the vehicle and the reaction to a motion command 
permit to find the limitation of the realization of the given 
command. The single link between the vehicle actuators and 
the road is the contact road - tire which means /x. Three 
different approaches based on this idea are already under 
investigations as [1] reported. Each approach focuses on a 
different level of the vehicle-road model to extract some 
laws about the road-tire friction. 

A vehicle-based system [2] and a wheel-based system [3] 
determine the slip value and use a tire dynamic model to 
extract /x. The relation between the slip and the friction 


coefficient is extracted from the Pacejka magic formula, 
which needs to be calibrated specifically for the tires. The 
single difference between both methodologies is the way to 
determine the slip value. The vehicle-based method uses 
measures of vehicle lateral and longitudinal motion, and 
wheel-based approach extracts the torque transferred on the 
wheels from the powertrain. 

The third method is based on measurement of tire de¬ 
flections from piezoelectric strips embedded in the regular 
tires of the vehicle. The inherent problem of this method 
comes from the local measurement on the tires and cannot 
give continuous data. Furthermore, the integration of several 
special sensors and their wireless communication to the 
body shield implies high cost. 

These three kinds of measurement are dealing with both 
road surface and tire. Therefore their accuracy can be 
relative good but can only provide reactive measures. In 
the next couple of years the assistant systems will have to 
deal wider with the environment. Therefore exterioceptive 
sensors will be needed to look at the /x coefficient in advance 
like camera or laser scanner. 

Unfortunately, as they have no tire information, they 
can only deliver a worth case estimation. Therefore the 
predictive measurements are up to now depending on a 
calibration of the tires’ model contrary to the reactive 
measurements, which could determine the characteristics 
of tires by themselves. With these predictive methods, 6 
different classes of /x can be defined : (1) icy, (2) wet, (3) 
humid cobblestone, (4-6) from bad to excellent coefficient. 
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Fig. 1. Ranges of /x 
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II. Algorithms using a camera 
A. Extraction of the ranges to analyze 

Some investigations with camera have already been done 
like in Japan by Kuno and Sugiura [4]. But these tests are 
based only on detection of the road surface gloss, which 
is not the only one factor for the estimation of the friction 
coefficient. Here the approach will try to go step further 
with a grayscale camera. 

The results of a lane detection like in [5] define the lateral 
limits of the range to analyze. In addition an algorithm for 
object detection defines the depth of the analysis range. No 
correction is needed to obtain a bird view like done in [6] 
because the picture will be split into several small areas 
like on figure 2 for local analysis with a low residual errors. 
To simplify the algorithms, a static height of 30 pixels is 
set for the different areas. This able to get maximally 9 
areas to monitor on the (640x480) picture. If the areas are 
not wide enough, the results of the computation will not 
be stable enough as some important variation of luminance 
may appear during some instants. 



Fig. 2. Split of the range to monitor 


B. Choice of the new methodology 

The main idea is to determine for each range above the 
vehicle the type of condition: road surface, humidity, snow 
etc. Three different approaches are already possible to use 
for the analysis: 

• Statistical approach: the luminance of the pixels and 
their repartition on histograms are analyzed to extract 
some properties. 

• Macroscopic approach: the co-occurrence matrix is 
calculated to describe the amount of neighborhoods of 
the pixels and the energy of the luminance. 

• Microscopical approach [7]: a 2D Fourier transforma¬ 
tion is applied on the picture to model the irregularities 
of the road and the regular patterns. 

As explained in [4], a robust use of the statistical 
approach needs a learning system. Due to the huge needs 
of computation capacity, it cannot be integrated now into 
embedded systems right now. The last approach based on 
the high frequencies of variation of road texture is quite 
difficult with the use of a camera, which is continuously 


vibrating due to the connection to the chassis of the 
vehicle. 

The macroscopic approach will be used here by 
integrating both texture analysis and gloss measurement. 
This approach will be partially similar as estimation of 
landscape characteristics based on snapshots taken by 
satellites like in [8], [9] or [10]. Some properties of this 
matrix like the global luminance, the energy or the variance 
can be used to characterize some types of textures. 

The co-occurrence matrix from the different ranges 
will be calculated and hyperspheres will be matched 
on the characteristically forms of the matrix. Next the 
hyperspheres are compared to the different models for the 
current luminance. Finally the new graph of /x over the 
distance is fused with the other graphs computed over the 
time to enhance the reliability. 

III. Predictive measures 
A. Co-occurrence matrices 

For a picture / with n levels of gray (here 50), its co¬ 
occurrence matrix Mj with the dimensions n x n will 
describe the amount of pair wise of pixel luminance on 
the picture. For each pixel pij with the index (ij) and the 
luminance k , its likelihood with its right neighbor qi.j+\ 
with a luminance l increases the value with the index 
(k, l) of the co-occurrence matrix. On the next figure, the 
co-occurrence matrices for a macadamized road and for 
cobblestones shown interesting differences. A summation 
of the pixels’ level on the diagonal top left to bottom right 
is corresponding to the histogram of the picture. And a 
summation on the other diagonal describes the histogram 
of the picture derivate. 



(a) Macademized road (b) Cobblestones 


Fig. 3. Example of two co-occurrence matrices 

The noticeable difference between both matrices can be 
used to estimate the type of texture. A 3D rendering of this 
matrix always looks like a Gaussian surface. Its maximum 
will be used as first characteristics because it corresponds 
to the global luminance. The width of the surface along the 
diagonal and the wise perpendicularly to it will be the two 
other characteristics. Here the height of the maximum will 
not been taken into account, even if it describes indirectly 
the homogeneity of the picture. But to accelerate the com¬ 
putation, Gaussian surfaces will not been used directly but 
only the definition of there ranges. 
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The form of these curves will be well defined for the 
road portions close to the vehicle as shown on figure 5, but 
the widest portions will get blurred matrices, because of the 
distance and the image correction. Therefore their matrices 
will tend to go to the white levels (bottom right) while good 
luminance or to the dark levels (up left) during night. 



Fig. 4. Co-occurrence matrix for a macadamized road (u = 0.65) 


B. Online extraction of the patterns 

Many measures have been performed on test tracks, where 
the friction coefficient has been estimated beforehand, with 
different conditions (dry, humid, wet) as on figure 5. The 
measures with snow have been done independently from 
the road texture because they are partially masked with the 
snow. 



Fig. 5. Co-occurrence matrix for cobblestone roads (yu=0.45) 


The measures have been matched together to define static 
models for several levels of luminance. The modification of 
the patterns over the luminance (or actually the position of 
the optimum) is quite linear between the two extremems 
shown on figure 6. With this approximation, the patterns 
can be extracted on-line by looking at the luminance. The 
position of the maximum of the co-occurrence matrix is 
searched on the diagonal and its distance to the two extreme 
weights there patterns. 



Fig. 6. Extreme patterns 


C. Single determination of the graph of \i 

Consequently a each time, for several areas to analyze 
there exists a rhombus to classify with different models. 
Each pattern has to be matched on the rhombus and the 
distances between the j extremities E 3 are measured. 
Parallel to the generation of the patterns, the standard 
deviations have also been measured to determine the 
probabilities of appurtenance as Gaussian curves. The 
confidence values for the j extremities EE can be 
extracted from these empirical curves and the fusion of 
these 4 inputs set the probabilities Pi of the i different 
classes of p with the following equation. 

P . __ if % • ^ 

n, a j ■ -P (-£■( jj+pf a j ■ 

The aj parameters weight the influence of each extremity. 
In practice, the extremities along the top left to bottom right 
diagonal have a higher weight because their wide range is 
easier to separate into different classes. The estimation of 
p is calculated from the fusion of the different probabilities 
by using the next equation. The final class of the friction 
coefficient is given after rounding the results. 

M = E Pi ' p* ( 2 ) 

i 

On the example shown on figure 7, the graph of p ahead is 
quite stable around 0.55, which is well corresponding to the 
road (gravel). But two degradations down to 0.3 (humidity) 
can be seen for the areas 5 (6m) and 8 (15m), they are 
corresponding to the parts with a high lighting. Here the 
systems thought it has seen wet parts of the road. Moreover, 
the maximal stability range is about 50 m in practice, which 
corresponds to the ranges bellow 12. 

D. Time correlation of the measures 

The confidence of the measure can be improved over the 
time by correlating the different measures. First the graph of 
p needs to be transformed to give p for all points ahead the 
vehicle. As the estimation is made only for regular areas on 
the picture, this graph will be constant steps with increasing 
size like on the figure bellow. 
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(a) Standard road on (b) Corresponding instantaneous graph 

test tracks 

Fig. 7. Example of two co-occurrence matrices 
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Therefore a reactive sensing method has been also inte¬ 
grated into the vehicle. This method is based on another 
analogy with the driver: the analysis of the sound coming 
from the road - tire contact. A small microphone (sampling 
rate mono 22050 kHz omni-directional) has been stuck on 
the fender of the front axle (at the opposite position of 
the motor) as shown on figure 9. At this position it can 
use optimally the loudspeaker effect [11] from the space 
between the road and the tire. 



Fig. 9. Position of the microphone 


Than the speed needs to be taken into account to match 
the new measures over the others like on the given figure. 
The range near to the vehicle will of course be the place 
with the most important number of measures. But all the 
measures have not the same confidence. Actually the first 
measures, which have been done as the vehicle was far 
from this point, are more roughly performed as the last 
ones. At 70 km/h about 50 measures can be matched on a 
same point for a visible analysis distance of 50 m. 

That is why the fusion algorithm needs to take into 
account a weight for each measure depending on the time 
when this measurement was performed. Therefore the 
measures made for a same point are sorted in clusters, 
maximally one per fi classes. Than these clusters can be 
weighted to select the most important class. Finally for 
this class and the two neighbored classes the mean value 
is calculated taking into account once again the weights of 
the measures. 

IV. IMPROVMENT OF THE RELIABILITY WITH A 
MICROPHONE 

A. Needs of a reactive measurement 

The predictive estimation can get some good results, but 
there can be somehow a need to estimate /i with another 
methodology, for example a reactive one. Indeed, critical 
systems need to have a redundancy of the sensing data 
to avoid failures due to problems with sensors or their 
algorithms for the analysis. 


B. Extraction of the characteristically frequencies 

To extract the interesting frequencies from all this noise, 
the band of frequencies between 100 Hz and 650 Hz is 
grabbed with a dedicated microphone. The low frequencies 
have to be filtered to avoid the measurement of the motor 
noise [12] and the noise generated from vehicles near from 
the sensor. Therefore the analysis cannot be performed at 
low speed (in practice bellow 40 km/h) but in such cases 
the analysis can be avoided because of the lower interest 
of the results: ABS [13] [14] and traction control are for 
example disable for speed lower than 20 km/h. 

As soon as the speed is high enough, some frequencies 
will grow up and generate the characteristically sound that 
the driver can hear too. Depending on the speed and on /i, 
there positions and amplitudes change and give the possi¬ 
bility to determine both speed and /i if enough frequencies 
are analyzed like on figure 10. Of course this analysis can 
be simplified if the speed is get from an external source. 

The detection of the interesting frequencies is based first 
on a detection of the local maximum each 20 Hz. The 
curve fitting these points is after that modified by a low- 
pass filter (depicted on the same figure). From this filtering 
are extracted the most important local maximums, which are 
also on the same figure. Starting on the locations, the closest 
and highest peaks are sought in there ranges. Unfortunately 
the amplitude of these frequencies goes down quickly and 
this phenomena limits hence the number of frequencies, 
which can be monitored. In practice 3 frequencies have 
been analyzed permanently and sometimes two others can 
be monitored too. 
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Fig. 10. Matching of the patterns at 70 km/h 


C. Extraction of the model 

First the peak corresponding to the sound of the motor 
(and especially the rounds per minute) has to be found 
from the list of potential interesting frequencies to de¬ 
fine the speed. The figure 11 shows the position of this 
frequency (J 7 ). After measures with the same speed but 
different rounds per minute ( rpm ), the following law has 
been established for the vehicle. This law is only acceptable 
is the speed is taken constant; as soon as the vehicle 
accelerates or brakes, this peaks will be moving. For the rest, 
the increasing of the speed has a unique effect to increase 
the perturbation noise as well. 



Fig. 11. Influence of the rpm for n = 0.65 
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Fig. 12. Influence of ^ at 70 km/h 


has been chosen instead of any other global matching as the 
models can changed drastically from one vehicle to another. 
This method avoids the mathematical modeling of the ranges 
or a new learning phase of a neural network. 

model O-O-O-O-O—* frequency 

t 'Z \t 

measure -O-O-O- ► frequency 

Fig. 13. Matching of extracted frequencies on a model 

To do this, Gaussian curves are fit on the different ranges 
in the models. From the distance of the peak amplitude to 
the optimum of each range, the probability to corresponds 
to this class is implicitly given. 

After that, for each class of /i, several probabilities 
are determined. The same approach for the fusion can 
be used as for the results coming from the camera by 
using the equations 1 and 2. But here the number of 
measures determined will change over the time, if the high 
frequencies are not extracted from the noise. 


T = 0.034 • rpm + 1 

The model is based also on several measures with differ¬ 
ent speeds and different friction coefficient like on figure 12. 
This model is constituted of many 30Hz wide rectangle, 
whom centre is placed at the calculated averages on the 
both axis and the height is equal to 3 times the variance 
of the amplitudes. The final results of the models can be 
shown on the figure 10. 

D. Estimation of the fi coefficient 

The amplitude of each frequency is analyzed separately to 
determine to which model it corresponds. This local method 


V. Integration of reactive and predictive 

MEASURES 

In normal situation the checking algorithm is looking at 
the consistency of the results coming from both sources of 
measurement. The comparison of the results can be done 
only once the vehicle has reach the point of measure. As 
long as the measures are corresponding or the predictive 
data are a little bit lower than the reactive data, the 
predictive way of measurement gets the priority. Indeed if 
the /i is underestimated, the vehicle safety will not be in 
danger as the vehicle dynamics will be just reduced. 

But if the predictive measures are bigger than the 
reactive ones, a dangerous conflict will appear. It may be 
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possible that a small road portion has been incorrectly 
analyzed if its surface is not wide enough. In such cases 
several types of textures have been matched together and 
the results tends to be estimated as a normal macadamized 
road (fi « 0.65) as it can emphasis most kind of local 
textures. This phenomenon can also happen if their has 
been dirt (dust or water) on the windshield. In such cases 
temporal errors have no real impact on the vehicle safety as 
their range are not wide enough to destabilize the vehicle 
for example. Therefore a counter is integrated into the 
checking algorithm and it will switch off the predictive 
measurement only after a distance of 20 m. 

VI. Conclusion and future works 

A predictive method of measurement based on a camera 
has been presented here. The extraction of the textures and 
the gloss through the use of co-occurrence matrices, permits 
to estimate fi with 6 different classes. Over the time an 
auto-correlation improves the reliability of the estimation 
by detecting inconsistencies. But the blur depending on 
the speed is not well taken into account and needs more 
investigation because of its impact on the picture. 

After that, a reactive method based on a microphone 
extracts characteristically frequencies and matches them on 
models. These patterns are depending on the vehicles and 
the tires. This method must be calibrated with a generic tire 
model. But it can deliver a measure of fi and not just an 
estimation. Furthermore, the life cycle of such devices may 
be too short for automotive industry as they are integrated 
where all the dust coming from the road is binding itself on 
the vehicle shield. 
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